Skip to content

fix: budget the real-CLI tests; destroy removes the emptied Project#106

Merged
wmadden merged 2 commits into
mainfrom
claude/ci-cleanup-and-test-budget
Jul 16, 2026
Merged

fix: budget the real-CLI tests; destroy removes the emptied Project#106
wmadden merged 2 commits into
mainfrom
claude/ci-cleanup-and-test-budget

Conversation

@wmadden-electric

Copy link
Copy Markdown
Contributor

Two unrelated fixes, both from the same incident — a dev workspace hitting planLimitReached.

1. The integration tests were mis-budgeted, not flaky

test/integration/test/cli.extension-config.test.ts spawns the real prisma-composer binary — which resolves both /control entries and evaluates a config — and neither test set a timeout, so both inherited bun test's 5000ms default. Real CI runs take 5148ms and 5747ms.

So they failed roughly two runs in three across unrelated PRs, and the symptom hid the cause: the process is killed before it writes, so you get

Expected to contain: "no built entry at"
Received: ""
  ^ this test timed out after 5000ms

which reads as a broken CLI, not a budget. Now 30s — they cost ~1.7s locally, and CI is ~3× slower under contention.

Worse than the flake itself: neither Test nor the deploy canary is a required check, so this was a red check people learn to ignore.

run-alchemy.test.ts also failed once in the same run (Expected: 0, Received: 1). Deliberately not touched: it spawns a trivial fake script, runs in <1s across repeated runs, and a timeout would surface as signal set / status: null, not a clean non-zero exit. Adding a timeout there would paper over a cause we haven't found. Worth a look if it recurs.

2. destroy --production now removes the app's Project

destroy tore down a stack's resources but the Project survived forever, so hand-run stacks piled up as empty Projects until the workspace hit its plan limit. The CI sweeper can't help — it only runs against CI's own workspace and only matches ^<prefix>-ci-\d+$.

This mirrors the Branch cleanup one level up: destroy --stage already deletes its Branch after a successful alchemy destroy (relying on the API refusing to delete a Branch with live members). So destroy --production now deletes its Project, with the API's own refusal as the guard.

Cleanup is best-effort and never throws — the destroy already succeeded, and failing the command over a cleanup step would be worse than leaving a shell:

Response Behaviour
204 / 404 Project removed — nothing was left in it
400 "existing dependencies" kept — it still holds another stage's resources
anything else warn, continue

The API contract differs from the assumption going in. I expected a 409, mirroring Branch delete. It isn't: deleteV1ProjectsById answers 204/400/401/404 — there's no 409 in that operation's schema at all; only branch delete has one. Verified against @prisma/management-api-sdk's types, and consistent with the existing postgres/Project.ts resource. So 400 is the kept case.

Users can observe this, so per .agents/rules/user-facing-surface-changes.mdc it lands in docs/guides/deploying.md and the skill in this PR.

Tests

  • container.test.tsdeleteProject issues the DELETE, tolerates 404, surfaces other errors.
  • ensure-containers.test.ts — success, 404, 400 keeps the Project and does not throw, other errors warn, missing token warns. The 400 case was mutation-checked: making it throw fails the test; reverted, green.
  • run.test.ts--production deletes the Project; --stage never touches it; a failed destroy skips it; deploy never touches it.
  • Fixed a regression this surfaced in four pre-existing --production tests that had no deleteProject seam and no token, so the real function's warning broke a not.toHaveBeenCalled() assertion.

Green: lowering 103, CLI 95, integration 2 (1.6s/1.7s). Typecheck, lint:deps, biome clean. No live deploy run.

🤖 Generated with Claude Code

wmadden-electric and others added 2 commits July 16, 2026 14:02
These two spawn the real prisma-composer binary, which resolves both /control
entries and evaluates a config, and neither set a timeout — so they inherited
bun test's 5000ms default. Real CI runs take 5148ms and 5747ms, so they failed
roughly two runs in three across unrelated PRs, and the symptom hid the cause:
the process is killed before it writes, so the assertion reports an empty
stderr rather than a timeout.

Not flaky — mis-budgeted. 30s of headroom; they cost ~1.7s locally, and CI is
about 3x slower under contention.

Signed-off-by: willbot <w.a.madden+machine@gmail.com>
Signed-off-by: Will Madden <madden@prisma.io>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
destroy tore down a stack's resources but the Project survived forever, so
hand-run stacks accumulated as empty Projects until the workspace hit its plan
limit. The CI sweeper cannot fix this: it only runs against CI's own workspace
and only matches ^<prefix>-ci-\d+$.

Mirrors the Branch cleanup one level up — destroy --stage already deletes its
Branch after a successful alchemy destroy, so destroy --production now deletes
its Project, and the API's own refusal is the guard.

Cleanup is best-effort and never throws: the destroy already succeeded, and
failing the command over a cleanup step would be worse than leaving a shell. A
Project still holding another stage's resources is kept.

Note the API contract differs from Branch delete, which was the assumption
going in: DELETE /v1/projects/{id} answers 400 "existing dependencies" — there
is no 409 in that operation's schema (only branch delete has one), so 400 is
the kept case.

Users can observe this, so per the user-facing-surface rule it lands in the
deploying guide and the skill in this PR.

Signed-off-by: willbot <w.a.madden+machine@gmail.com>
Signed-off-by: Will Madden <madden@prisma.io>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@pkg-pr-new

pkg-pr-new Bot commented Jul 16, 2026

Copy link
Copy Markdown

Open in StackBlitz

npm i https://pkg.pr.new/@prisma/composer@106
npm i https://pkg.pr.new/@prisma/composer-prisma-cloud@106

commit: fcec1c1

@wmadden wmadden merged commit ac1e7b1 into main Jul 16, 2026
15 checks passed
@wmadden wmadden deleted the claude/ci-cleanup-and-test-budget branch July 16, 2026 12:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants